
METHOD OF DESCRIBING MULTIPLE LEVEL DIGEST SEGMENT 
INFORMATION SCHEME FOR MULTIMEDIA CONTENTS AND APPARATUS 
FOR GENERATING DIGEST STREAM FROM DESCRIBED MULTIPLE LEVEL 
DIGEST SEGMENT INFORMATION SCHEME AND METHOD THEREOF. 
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BACKGROUND OF THE INVENTION 
1 . Field of the Invention 

The present invention relates to a technique for describing a multiple level 
10 digest segment information scheme for multimedia contents in a content-based 
data area of a multimedia stream and method for generating a digest 
streams(highiights) using the multiple level digest segment information scheme for 
the purpose of providing a multiple levels of digest streams for a multimedia 
stream. 
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2. Description of the Prior Art 

A digest stream for a multimedia stream is provided for the purpose of 
enabling users to understand the overall story and the structure of a multimedia 
content or to find and move to desired position fast. 
20 Multimedia server system such as a VOD(Video-on-Demand) server system, 

a broadcast system, a karaoke, etc., can provide digest streams(highlights) for 
multimedia contents (e.g., movie, drama, sports, etc.) to achieve above purposes. 

A digest stream(highlight) of a multimedia stream is constructed by 
25 arranging audio-visual segments, each of which is the essence of an original 
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stream in the development of the story. Here, the segment is a sequence of 
continuous frames. Since the digest stream is a sub stream of the multimedia 
stream, the size of the digest stream is much smaller than that of the original 
stream. 

Therefore, a user who want to understand the overall structure and/or the 
story of a multimedia content prefers to access a digest stream rather than 
accessing the entire stream. The digest stream also can be used as a program 
guide for users who want to select a program from a multimedia database. 

There are two types of summarization method for multimedia contents. The 
one is providing key frames or key regions and the other is providing digest 
streams(highlights) which is constructed by gathering and arranging important 
segments of an original stream 

In the former method, since it is difficult for a user to understand the entire 
flow of the original sueam and it is difficult to display too many key frames at a 
time, an additional operation such as a screen scrolling is required in order to 
understand the content of the stream only with the key frames, thereby causing an 
inconvenience. 

In the latter method, generally, a content provider extracts a sub stream , 
from the original stream, and stores it separately. In this case, an additional, 
storage for storing the digest stream is required. In addition, the content provider 
edits only a certain time amount of the digest stream and provides it to the user. 
Accordingly, the user can access only a single level digest stream edited by the 
content provider. Therefore, users cannot obtain various digest streams of the 
desired length. Suppose that a content provider provides only a 10 minute 
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highlight for a 2 hour video. Under that situation, users may want to access 5, 10, 
20 minute digest streams respectively, depending upon their situation. In that case, 
a user who want 5 or 20 minute highlight for the given content can not browse the 
digest stream of their desire. Only 10 minute digest stream will be served 
5 regardless of the user's request. Therefore, in order to satisfy user's various needs, 
the content provider has to edit and store multiple levels of digest streams for one 
multimedia stream. In this case, a large amount of storage is required to store 
multiple versions of digest streams, and accordingly, there occurs difficulty in 
Q managing each of the digest stream. 
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tj\ SUMMARY OF THE INVENTION 
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£3 Accordingly, it is an object of the present invention to provide a multiple 

Til levels digest streams, thereby dynamically coping with the level of user's need 

CO 

£3 15 with resnent tn a d»gest stream. 

It is another object of the present invention to provide multiple levels of 
digest streams with small amount of storage. 

It is yet another object of the present invention to reduce the encoding time 
of a digest stream. 

20 To achieve the above objects, there is provided a method of describing 

information about multiple levels of digest streams for multimedia contents in 
accordance with an embodiment of the present invention which includes tha steps 
of: describing the level information of digest segments by multiple levels in the 
content-based data area of the multimedia content; describing the digest level 

25 information of each digest segment and the time range information in a digest 
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segment information structure; and describing multi level digest segment 
information scheme with digest segment information structures. 

To achieve the above objects, there is provided a method of describing 
information about multiple levels of digest streams for multimedia contents in 
5 accordance with the second embodiment of the present invention which includes 
the steps of: describing the digest level information of digest segments to a digest 
level header by multiple levels in the content-based data area of a multimedia 
stream; describing the time range information of each digest segment in a digest 

Q 

q segment information structure; and describing information about digest streams by 

10 arranging the digest segment information structures of the same digest level. ; 

in describing multi level digest segment information scheme with digest level headers 

□ 

L| each of which has a list of digest segment information structures. In order to 

j:3 achieve the above objects, there is provided a method of generating multiple 

nj levels of digest streams for multimedia contents in accordance with the present 

m 

£3 15 invention which includes the steos of Hetecting the digest level and time range 
information of each digest segment information structures from the multiple level 
digest information scheme contained in the content-based data area of a 
multimedia stream; and, when a condition is queried by the user, generating a 
multiple level digest stream by arranging the digest segments with a priority of. 

20 more than a certain level corresponding to the condition in a time sequence. 

In addition, to achieve the above objects, there is provided an apparatus for 
generating multiple levels of digest streams for multimedia contents in accordance 
with the present invention which includes an user input unit; a digest stream level 
determining unit for determining a digest level of a digest stream corresponding to 

25 the condition queried by the user input unit and outputting the same, upon. receipt 
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of a multimedia stream signal; and a decoder for decoding the digest segments 
having the above digest level and outputting the same. 

Additional advantages, objects and features of the invention will become 
more apparent from the description which follows. 

BRIEF DESCRIPTION OF THE INVENTION. 

The present invention will become more fully understood from the detailed 
description given herein below and the accompanying drawings which are given 
by way of illustration only, and thus are not limitative of the present invention, and 
wherein: 

Figure 1 shows an example of digest segments for a multimedia stream in 
accordance with an embodiment of the present invention; 

< 

Figure 2 shows the construction of a multiple level digest segment 
information schemer uf a muiiimedia stream in accordance with an embodiment of 
the present invention; 

Figure 3 shows the construction of a multiple level digest segment 
information scheme of a multimedia stream in accordance with another 
embodiment of the present invention; 

Figure 4 shows an example of multiple level digest streams constructed by 
the multiple level digest segment information scheme; 

Figure 5 is a graph showing the total running time of digest streams 
according to digest levels; 

Figure 6 is a schematic block diagram of an apparatus for generating 
multiple levels of digest streams for multimedia contents in accordance with an 
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embodiment of the present invention; 

Figure 7 is a schematic block diagram of an apparatus for generating 
multiple levels of digest streams for multimedia contents in accordance with 
another embodiment of the present invention; 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS. 

In a content-based data area of a multimedia stream, generally, an 
appearance of an audio-visual object, occurrence information of each event, state 

10 information of audio-visual objects, events, backgrounds, scene information, shot 
information, segment information, etc. are described on a time axis. According to 
the present invention, the data structure of a multiple level digest segment 
information scheme for providing multiple levels of digest streams is described in 
the content-based data area thereof. The multiple level digest segment information 

15 schema is constructed by a plurality uf multiple level digest segment information 
structures, each of which is a data structure for describing digest segment in which 
a time range information of the digest segment is described and also a digest level 
information can be described. Moreover, in some cases, data for representing the 
running time of a digest stream or the total running time of digest segments of 

20 each digest level can be additionally described. A digest stream is comprised of 
digest segments. An importance level associated with each digest segment is 
used as a digest level for generating multiple levels of digest streams dynamically. 

In addition, each digest segment information structure has a time range 
information of time continuous data, and the time range information can be 

25 represented by the start point and end point or the start point and duration of the 
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digest segment. 

Meanwhile, a digest segment is different from a shot, a scene and the like. 
Taking example by drama, generally, a digest stream is not constructed by 
connecting shots or scenes. Because the whole content can be analogized by 
browsing only parts of a shot or a scene in most cases, a digest stream is 
generally constructed by connecting some parts of shots or scenes. In a field such 
as sports, digest stream does not use entire shot or entire scene in order to 
construct a digest stream for the reason of the characteristics thereof. 

In some cases, the digest segment corresponds to a shot or a scene, but, in 
most cases, it does not necessarily correspond to any of them. Therefore, in the 
present invention, the multiple level digest segment information scheme for 
providing multiple levels of digest streams by assigning an importance level to 
each digest segment, rather than by assigning an importance level to a shot or a 
scene, is proposed. Here, the digest segment is defined as a sequence of 
cuimi iuuus frames, wnich is a general idea of covering shots or segments. 

The construction of a digest stream in accordance with the present 
invention will now be described with reference to Figures 1 through 5. 

Figure 1 shows an example of digest segments for a multimedia stream in 
accordance with an embodiment of the present invention. As illustrated in Figurel, 
the horizontal axis represents the time axis, and the vertical axis denotes the 
digest level axis. It is shown that an original stream, which is a multimedia stream, _ 
is divided into a plurality of digest segments, which are divided into a plurality of 
digest levels. In the present invention, it is assumed that a first digest level '0' is 
the highest in priority, and a n-th digest level 'n-1' is the lowest. 

Figure 2 shows the construction of a multiple level digest segment 



information scheme of a multimedia stream in accordance with an embodiment of 
the present invention. A portion illustrated by a square is a digest segment 
information structure, and a number illustrated within the square is an ID of a 
digest segment information structure, which is stated only for the convenience of 
5 explanation. Digest segment structures can be sequentially arranged in a time 
sequence, each of which has its digest level and time range information(start point 
and end point or start point and duration). 

Figure 3 shows the construction of a multiple level digest segment 
information scheme for a multimedia stream different from that of Figure 2. Digest 

10 level headers(L 0l L 1t L n ^) are divided according to digest levels, and assuming 
that the number of digest levels is n, the number of digest level headers is n. For 
each digest level header(L 0 , L 1f L^), digest segment information structures of 
the corresponding digest level are arranged. For convenience, the digest segment 
information structures can be arranged in a time sequence. For instance, a first 

is digest !svc! hcader(L 0 ) lias digest segment information structures 5, 10, 30 and 
138 arranged therein, and a second digest segment information structures 12, 22, 
40, 50, 54, 70 and 88 arranged therein. In this manner, unlike the digest segment 
information structure in accordance with an embodiment of the present invention 
as shown in Figure 2, the digest segment information structure in accordance with 

20 another embodiment of the present invention as shown in Figure 3 has its time 
range information, and its digest level is described in its digest level header. 
Herein, information on the position of a first digest segment information structure 
among a plurality of digest segment information structures belonging to each 
digest level header may be needed. In this case, the information on the position 

25 can be contained in the digest level header or other area. Meanwhile, the 




presentation as illustrated in Figure 2 and the presentation as illustrated in Figure 
3 are easily convertible to each other. 

The method of obtaining multiple levels of digest streams from a multiple 
level digest segment information scheme for a multimedia content thus described 
5 will be illustrated below. 

It is assumed that the number of digest levels for the original stream is n, 
and the number of digest segments for the original stream is m as described 
above. Sj denotes the j-th digest segment information structure, L(Sj) denotes the 
digest level of the digest segment information structure S,, and Hj denotes the 
10 digest stream whose digest level is i. The digest stream can be represented by a 
sequence of digest segment information structures, so that when a set of digest 
segments of the digest stream(H i ) is denoted by S(Hj), the set S(Hj) is defined as 
follow: 

SfH^MSj I L(S,) = 0,1,2,. .J, G=0,1,2,...,m-1)} 

15 That is, ecch element of the set S(Hj) is a digest segment information 

structure whose digest level is less than or equal to i. All the digest segment 
information structures contained in the set S(Hj) can be sorted in a time sequence. 
According to the definition of the above set S(Hi), the digest stream whose digest 
levels is k+1(H k+1 ) includes the digest stream whose digest level is k(H k ). 

20 Taking example by Figure 3, the first digest stream(H 0 ) is constructed by 

arranging digest segment information structures(S5, S10, S30 and S138) of digest 
level '0' in a time sequence, and the second digest stream(H 1 ) is constructed by 
arranging the digest segment information structures(S5, S10, S30 and S138) 
digest level '0' and digest segment information structures(S12, S22, S40, S50,.. 

25 S54, S70 and S88) of digest level T in a time sequence. That is, the second 



digest stream(H 1 ) is constructed by arranging digest segment information 
structures(S5, S10, S12, S22, S30, S40, S50, S54, S70, S88 ana S138) in a time 
sequence. 

Figure 4 shows an example of multiple level digest streams constructed by 
the above-described method. 

Next, the running time of each digest stream can be obtained based on the 
time range information contained in each of the plurality of digest segment 
information structures, which will be described below. 

The total running time of the digest segment information structures . 
belonging to a i-th digest level is the sum of the running time of each of digest 
segment information structures which belong to the i-th digest level. 

That is, the running time R(H k ) of the digest stream whose digest level is 
k(H k ) can be represented by the sum of the running time of digest information 
structures whose digest level is less than or equal to k, can be expressed as 

R(HJ = § /?(£/) 

Where R( Li) denotes the sum of the running time of each digest segment 
information structure whose digest level is i. 

Figure 5 is a graph showing the total running time of a digest streams 
according to digest levels. As illustrated therein, since the digest stream whose 
digest level is k+1(H k+1 ) includes the digest stream whose digest level is k(H k ). the 
running time R(H k+1 ) of the digest stream(H k+1 ) is longer than or equal to the 
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running time R(H k ) of the digest stream(H k ). 

Next, the operation of providing a digest stream according to the query of 
the user will be described. 

Figure 6 is a schematic block diagram of an apparatus for generating 
5 multiple levels of digest streams for a multimedia content in accordance with an 
embodiment of the present invention. In the apparatus as illustrated in Figure 6, 
when a plurality of running times is displayed, one of which is queried by the user, 
a digest stream corresponding to the queried running time is provided. 

As illustrated in Figure 6, the apparatus in accordance with an embodiment 
10 of the present invention includes a digest stream level determining unit 1 for 
determining and outputting the level of a digest stream corresponding to the 
running time queried by the user, upon receipt of a multimedia stream signal; a 
user input unit 2 for outputting the running time queried by the user to the digest 
stream level determining unit 1 by the operation of the user; a decoder 3 for 
15 cutpuUifiy digest segments having a digest level by decoding based on the digest 
level which is inputted from the digest stream level determining unit 1, upon receipt 
of a multimedia stream signal; and a browsing/recording/editing unit 4 for 
sequentially browsing, recording, and editing the outputted digest segments. 

The operation of the apparatus in accordance with an embodiment of the 
20 present invention thus described will now be described. Meanwhile, the operation 
of browsing, recording and editing performed by the browsing/recording/editing 
unit 4 is carried out by a well-known technique of the conventional art, and 
accordingly a detailed description thereof is omitted. 

The digest stream level determining unit 1 includes a digest segment 
25 information scheme analyzing unit 11 and a digest stream information display unit 
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12. A content provider or a media file supplies the digest stream level determining 
unit 1 with a stream signal, and the digest stream level determining unit 1 
determines a digest level of a user-selected digest stream. Herein, the multimedia 
stream signal includes a stream data and a content-based data for the stream, and 
the content-based data of the stream includes related multiple level digest 
segment information scheme. 

The multiple level digest segment information scheme analyzing unit 11 
computes the running time of each digest stream by analyzing the multiple level 
digest segment information scheme. The digest stream information display unit 12 
displays the running time by computed digest streams to the outside. 

When the user inputs a desired running time among the displayed running 
times, the running time is queried through the user input unit 2. The digest stream 
information display unit 12 determines a digest level of a digest stream 
corresponding to the queried running time, and outputs it to the decoder 3. The 
decoder 3 outputs digest segments with the digest level by decoding based on the 
digest level which is inputted from the digest stream level determining unit 1, upon 
receipt of a multimedia stream signal. The outputted digest segments are 
sequentially browsed, recorded, and edited by the browsing/recording/editing unit 
4. 

Taking example by Figure 4, if the digest level queried by the user is "1", the 
decoder 3 outputs digest segments of digest level "0" or "1" from the multimedia 
stream signal by decoding. 

In this manner, only the digest level of a digest stream is inputted to the 
decoder 3, and the decoder 3 selects digest segments from the multimedia stream 
signal based on the digest level and outputs them to the 
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browsing/recording/editing unit 4. 

Figure.. 7 is a schematic block diagram of an apparatus for generating 
multiple levels of digest stream for a multimedia content in accordance with 
another embodiment of the present invention. The apparatus as shown in Figure 7 
provides a digest stream with a running time most close to the running time 
queried by the user As illustrated in Figure 7, the apparatus in accordance with 
another embodiment of the present invention includes a digest stream level 
determining unit 5, an user input unit 2, decoder 3, and a 
browsing/recording/editing unit 4. The digest stream level determining unit 5 
includes a digest segment information scheme analyzing unit 11 and a digest level 
determining unit 51. Herein, the digest segment information scheme analyzing unit 
11, decoder 3, and browsing/recording/editing unit 4 are identical with those 
illustrated in Figure 6. 

When the user inputs a desired running time by the user input unit 2, the 
digest love! determii iii iy unit 5 i outputs the digest level of a digest stream with a 
running time most close to the running time inputted by the user. That is, the digest 
level determining unit 51 compares the running time queried by the user with the 
running time of a digest stream of each multiple level digest stream, selects a 
running time R(H k ) which is most close to the queried running time, and outputs a 
digest level(k) of the digest stream with the selected running time R(H k ) to the 
decoder 3. In Figure 5, the running time queried by the user is marked by a thick 
line. The running time corresponding to the digest level(k) is slightly longer than 
the queried running time, and the running time of a digest stream corresponding to 
a digest level(k-l) is slightly shorter than the queried running time. 

The decoder 3 receives the multimedia stream signal, and outputs digest 
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segments of a digest level lower than k or k-1 , based on the digest level(k or k-1 ). 
The outputted digest segments are sequentially browsed, recorded and edited by 
the browsing/recording/editing unit 4. 

Meanwhile, the embodiments of the present invention described above, in 
5 detail, the user input unit 2 as illustrated in Figures 6 and 7 is described by taking 
an operation of querying a running time of a digest stream for example, for the 
convenience of explanation. However, the user can queries other conditions 
besides a running time, the user input unit 2 can receive the conditions from the 
user, and the apparatus in accordance with the present invention provides a digest 
10 stream according to various conditions inputted by the user, which will be 
described below. 

When the user inputs various conditions, for example, an occurrence of 
events, persons, backgrounds, objects, and some kinds of situation information, 
the digest stream level determining unit 5 constructs a digest stream based on the 
iz condiLiunb, for example, occurrence of a events, appearance of persons, 
backgrounds, objects, and some kinds of situation information. The various 
conditions queried by the user are, for example, as follows: 



According to these queries from the user, the digest stream level 
determining unit 5 construct a digest stream based on a running time and various 
query conditions or filtering conditions. By lowering or raising the digest level of 
each digest segment, a digest stream is constructed. For instance, although a 
25 digest segment has a low digest level(with a digest tevel value close to n), the 



Digest "Titanic" movie to 10 minutes focusing on "Leonardo Dicaprio". 



Digest "Titanic" movie to 10 minutes focusing on sinking scene. 
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Digest a music to 100 seconds focusing on piano. 
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digest segment is considered as a digest segment with a little higher digest level 
for thereby constructing a digest stream if it corresponds to a query condition(e.g., 
"sinking scene"). Occurrence of events, appearance of persons, backgrounds, 
objects, and some kinds of situation information mentioned above are information 
generally contained in a content-based data area of a multimedia stream. 

Meanwhile, there is a need for rapidly accessing only the running time 
information of a possible digest stream for the convenience of applications. For 
this purpose, a digest level running time information or a digest stream running 
time information can be added to a primary digest segment information scheme as 
shown in Figure 2 or Figure 3. Herein, the digest level running time information is 
information about the sum of the running times of digest segments of the same 
digest level is described according to digest levels, and the running time 
information of the digest stream is the running time information of a possible digest 
stream. 

Such running time infuimaiion can be described using an extra data 
structure, for example, in the case of a multiple level digest segment information 
scheme as shown in Figure. 2. In the case of a multiple level digest segment 
information scheme as shown in Figure 3, the running time information can be 
described in each digest level header, or described using other data structure. 

The above-described digest level running time information or digest stream 
running time information can be used for a rapid running time computation and for 
a parity check for a damaged data. 

The parity check is possible by comparing the total running time of digest 
segments of the same digest level with the digest level running time of that digest 
level, and also by comparing the total running time of digest segments 
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corresponding to a digest level lower than a particular digest level with the digest 
stream running time of that digest level. More specifically, if the total running time 
of the digest segments of the same digest level and the digest level running time 
of that digest level are equivalent each other, it is determined that there is no data 
damage, and if the two running times are different, it is determined that there is a 
data damage. In addition, the total running time of digest segments having a digest 
level lower than a particular digest level and the digest stream running time 
corresponding to that digest level are equivalent each other, it is determined that 
there is no data damage, and if the two running times are different, it is determined 
that there is a data damage. 

Meanwhile, in the case that only the information about a possible digest is 
queried by a digest stream browser, the corresponding information can be 
displayed by accessing only the digest level running time information or the digest 
stream running time information. In the case that the user determines a digest 
!evc! from the irifuii nation, a digest stream can be generated by accessing only 
the actual digest segment. 

In this way, the digest level running time information and digest stream 
running time information additionally described in the multiple level digest segment 
information can be utilized for a rapid digest stream information transmission and 
for a parity check for determining whether or not there is a data damage.. 

As described above in detail, in the present invention, a multiple level digest 
segment information scheme is stored in the content-based segment information 
scheme of an original stream, and an importance level(i.e., digest level) of a digest 
level information, a time range information, etc. is given to a digest segment of the 
multiple level digest segment information scheme, whereby the amount of storage 
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for storing a digest stream is greatly decreased and it takes a very little time to 
encode each digest stream. 

Moreover, by dynamically generating a digest stream using a digest 
segment information scheme during an execution of an application, it is 
5 unnecessary to edit a digest stream and encode an actual data so as to provide a 
multiple level digest stream. Therefore, it is possible to dynamically provide a 
digest stream corresponding to the dynamically changing level of user's need for a 
digest stream while the amount of data is not increased largely Also, a user can 
browse a digest stream of a desired level for a desired time by using a multiple 

10 level digest stream provided. 

In addition, although it is explained above that a digest stream of a 
multimedia stream is provided for an user in a multimedia server system, the user 
can also understand the overall story and/or the structure of a multimedia content - 
and.he or she can find and move to desired position fast by accessing the digest 

15 stream when he or she has a CD(compact disc) for the content. 
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